A Comparison of Analysis of Covariate-Adjusted Residuals and Analysis of Covariance

Various methods to control the influence of a covariate on a response variable are compared. In particular, ANOVA with or without homogeneity of variances (HOV) of errors and Kruskal-Wallis (K-W) tests on covariate-adjusted residuals and analysis of covariance (ANCOVA) are compared. Covariate-adjusted residuals are obtained from the overall regression line fit to the entire data set ignoring the treatment levels or factors. The underlying assumptions for ANCOVA and methods on covariate-adjusted residuals are determined and the methods are compared only when both methods are appropriate. It is demonstrated that the methods on covariate-adjusted residuals are only appropriate in removing the covariate influence when the treatment-specific lines are parallel and treatment-specific covariate means are equal. Empirical size and power performance of the methods are compared by extensive Monte Carlo simulations. We manipulated the conditions such as assumptions of normality and HOV, sample size, and clustering of the covariates. The parametric methods (i.e., ANOVA with or without HOV on covariate-adjusted residuals and ANCOVA) exhibited similar size and power when error terms have symmetric distributions with variances having the same functional form for each treatment, and covariates have uniform distributions within the same interval for each treatment. For large samples, it is shown that the parametric methods will give similar results if sample covariate means for all treatments are similar. In such cases, parametric tests have higher power compared to the nonparametric K-W test on covariate-adjusted residuals. When error terms have non-symmetric distributions or have variances that are heterogeneous with different functional forms for each treatment, ANCOVA and analysis of covariate-adjusted residuals are liberal with K-W test having higher power than the parametric tests. The methods on covariate-adjusted residuals are severely affected by the clustering of the covariates relative to the treatment factors, when covariate means are very different for treatments. For data clusters, ANCOVA method exhibits the appropriate size. However such a clustering might suggest dependence between the covariates and the treatment factors, so makes ANCOVA less reliable as well. Guidelines on which method to use for various cases are also provided.